TITLE OF THE INVENTION 

BLIND SIGNAL SEPARATION SYSTEM AND METHOD, BLIND SIGNAL 
SEPARATION PROGRAM AND RECORDING MEDIUM THEREOF 

BACKGROUND OF THE INVENTION 

1. Technical Field to which the Invention Pertains 

[00001] The present invention relates to a blind signal separation technique, 
and more particularly to a blind signal separation system and method, a blind 
signal separation program and a recording medium thereof, in which a source 
signal is separated from a mixture signal into which the source signal with a time 
delay is mixed. 

2. Description of the Related Art 

N [00002] In a blind signal separation problem, there is a blind deconvolution 
method especially as a method for separating a signal mixed with a time delay. 
Zhang et al. proposed a method for learning a linear filter to minimize the KL 
divergence (Kullback-Leibler divergence) to an output signal, which is used as a 
measure of statistical independence for an observed signal into which plural 
independent signals are mixed temporally and spatially, thereby making a 
multi-channel blind signal separation. } 

[00003] On the other hand, there is a method using a learning algorithm based 
on overcomplete representations in the case where the number of independent 
signals is more than the number of observed signals. Olshausen et al. proposed a 
sparse coding network for learning to minimize an evaluation function which is 
defined by a square error between an input signal and an estimated model and the 
sparseness of an output signal. Also, according to Lewicki et al., the blind signal 

- 1 - 



separation is achieved by maximizing a posteriori probability of input data based 
on a maximum likelihood estimation. 

[00004] A learning algorithm proposed by Lewicki et al. will now be outlined 
below. When a source signal s = {si, .., s n } that is an n-dimensional independent 
signal is mixed into an m-dimensional mixture signal x = {xi, .., x m }, the source 
signal is formulated as follows. 

x=As (formula 1) 

[00005] In the formula 1, A is an mxn matrix (basis matrix). Considering 
each column of the basis matrix A as a basis function, each element of s is a 
coefficient (basis coefficient) of each basis function. Moreover, supposing that 
m<n, the mixture signal x is represented as a linear sum of overcomplete basis. 
The purpose here is to estimate the optimal basis matrix A and the source signal s 
from only the information of the mixture signal x. The optimal basis matrix A 
estimated here is the mixture matrix for mixing the source signal s to produce the 
mixture signal x. 

[00006] In the following, an estimation method for solving them will be 
described from the viewpoint of probability theory. First of all, estimation of the 
basis coefficients is made. Since the basis is overcomplete, s satisfying the 
formula 1 is not uniquely determined. Thus, the optimal s is acquired by 
maximizing the posteriori probability P(s I x,A) of s. This is achieved by solving a 
linear programming problem as follows. 

min c T I s I , subject to As = x (formula 2) 
Where c = (1, .., 1), and the objective function of linear programming is c T I s I = 
Zk I Sk I . This is equivalently achieved by maximizing a priori probability 



distribution P(s) under the condition As = x. It is assumed that P(s) is a 
Laplacian distribution with sparseness as follows. 

P(sk)ocexp(- 6 I Ski ) 
Where 6 is a parameter for deciding the variance. 

[00007] A learning algorithm for finding the most adaptable basis to the data 
structure is derived below. Here, a logarithmic posteriori probability for certain 
data x 



is regarded as a likelihood function, and the optimal basis is derived based on the 
maximum likelihood estimation that acquires A with the maximal likelihood. The 
maximum likelihood value A is searched by a gradient method by learning so that 
a derivative 

3logP(xl A)ldA 

is zero. 

[00008] However, the integration calculation of formula 3 to acquire P(x I A) is 
typically difficult, and its value can not be specifically obtained. Hence, an 
approximation expression that is obtained by expanding P(x I A) around A s by a 
saddle point method is employed to obtain 

/L 1 

logP(xl A) « const. + log P( A s)- — (x- A A s) 2 - — logdetH (formula 4) 

Here, 

X = 1 / o 2 , H = AA T A- V s V s log P( A s) 

Where a denotes a standard deviation of noise (x-As). A s is a solution in the 
formula 2. A learning rule is obtained from a derivative of logP(x I A) by A. If it 
is supposed that 




(formula 3) 
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V = d/dA 

the learning rule is given by the following expression (e.g., refer to "Learning 
overcomplete representations." M.S.Lewicki and T.J.Sejnowski, ,Neural 
Computation, Vol. 12). 

AA= AA T Vlog P(x I A) (formula 5) 
w - A(^ A s T + I) (formula 6) 

Here, 

z k =dlogP( A s k )/ds H 

[00009] Using the learning rule AA as obtained above, the basis matrix A is 
corrected by learning according to the following procedure. Correction with the 
learning rule AA is made for each element of the basis matrix A immediately 
before correction. 

(1) When the mixture signal x is the input signal, s is obtained from the input 
signal x and the basis matrix A by the linear programming method of the formula 

2. 

(2) AA in the formula 6 is calculated using s obtained according to the 
procedure (1), and the basis matrix A is corrected by the following expression. 

Anew =A old + 77 AA (formula 7) 
Where A new and A old denote basis matrixes before and after correction, 
respectively, and T| is a learning ratio. 

(3) The above procedures (1) and (2) are repeated until the basis matrix A 
converges. A converged value of the basis matrix A is calculated as a mixture 
matrix. Also, a solution A s in the formula 2 when the basis matrix A takes the 
converged value is an estimated source signal. 



[00010] However, in the conventional techniques, because a linear filter is 
employed in the algorithm of the blind deconvolution method, there is a limitation 
that the independent signals can not be extracted beyond the number of mixture 
signals. 

[00011] Also, in the signal separation algorithm based on overcomplete 
representations as proposed by Lewicki et aL, there is a problem that it is difficult 
to deal with the signal (temporally and spatially mixed signal) into which the 
source signal is mixed with a time delay, although the independent signals can be 
obtained beyond the number of mixture signals. The reason of this is as follows. 
[00012] For example, a sinusoidal time series signal is considered. In the 
sinusoidal wave, only if two basis functions (e.g., sin0 and cosG) with different 
phases are prepared, the sinusoidal wave with arbitrary phase can be represented 
in a linear combination of them. This indicates that the sinusoidal wave with 
different phase lies on the two-dimensional plane, irrespective of the number of 
sampling points. 

[00013] However, the typical signal waveform is rarely contained in such small 
dimensions. For example, considering a signal waveform such as a delta function, 
all spaces created by the sampling points are covered as the time passed, and it is 
impossible to represent the time lag by a linear combination of a small number of 
different bases. This always happens with the aperiodic waveform. 
[00014] From the above discussion, it is required to prepare the bases 
corresponding to all the time lags to deal with the signal mixed temporally and 
spatially. That is, this requires a great number of bases given by the number of 
sampling points x number of kinds of bases. In this case, each basis for the same 



kind of signal waveform is simply the signal with time lag which should have 
similar figure. 

[00015] When the signal separation algorithm using the overcomplete basis as 
proposed by Lewicki et al. is directly applied, it is still possible but very difficult to 
extract such a great number of bases by learning. 

SUMMARY OF THE INVENTION 

[00016] The present invention has been achieved to solve the above-mentioned 
problems associated with the conventional techniques, and it is an object of the 
invention to provide a blind signal separation system and method, a blind signal 
separation program and a recording medium thereof, in which the source signals 
are separated from the mixture signals into which the source signals are mixed 
temporally and spatially, when the number of source signals is more than or equal 
to the number of mixture signals. 

[00017] In order to achieve the above object, in the present invention, first of all, 
the mixture signals into which the source signals are mixed temporally and 
spatially are input. Then, the input mixture signals are formulated as an 
operation expression with a basis matrix composed of plural small matrixes which 
are the bases with time symmetry. Then, this invention applies a learning 
algorithm based on overcomplete representations as proposed by Lewicki et al. 
[00018] In this invention, when the learning algorithm based on overcomplete 
representations is applied, only the bases located in the middle column of the basis 
small matrixes composing the basis matrix before correction are corrected in 
accordance with the formula 7. And in the present invention, the new small 



matrixes composed of the bases located in the middle column after correction and 
plural vectors with the bases shifted successively in phase are created. 
[00019] And in the invention, the learned basis matrix is constructed based on 
the newly created small matrixes. In the present invention, it is determined 
whether or not the basis matrix has fully converged. If the basis matrix has fully 
converged, a converged value of the basis matrix is calculated as a mixture matrix, 
and the source signals are estimated. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[00020] Fig. 1 is a block diagram showing the configuration of a system 
according to an embodiment of the present invention; 

Fig. 2 is a chart showing an input signal and the bases; 

Fig. 3 is a diagram showing a network model representing a way of making 
a convolution; 

Fig. 4 is a diagram representing a temporally and spatially mixed signal x(t) 
as a linear sum of bases; 

Fig. 5 is a diagram showing a way of applying the filters to the source 
signals; 

Fig. 6 is a diagram showing the source signals; 
Fig. 7 is a graph showing a filter used for convolution; 
Fig. 8 is a chart representing the temporally and spatially mixed signals; 
Fig. 9 is a flowchart showing an example of a blind signal separation 
process; 

Fig. 10 shows an operation expression for formulating the mixture signals; 
Fig. 11 shows the basis small matrixes Aij; 



Fig. 12 shows the data configuration of the source signals sj; 
Fig. 13 shows the bases to be corrected; 
Fig. 14 shows a learned basis matrix; 
Fig. 15 is a graph showing a basis obtained by learning; 
Fig. 16 is a chart showing the estimated source signals; and 
Fig. 17 is a table showing the correlation coefficients between the source 
signals and the estimated source signals. 

Fig. 18 is a diagram showing formula 12 and formula 13. 
Fig. 19 is a diagram showing formula 14. 
Fig. 20 is a diagram showing basis matrix A. 
Fig.21 is a diagram showing basis small matrix. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[00021] As shown in Fig. 2, to suppress temporal movement of components 
constituting an input signal, the bases each having the same shape but different 
phase are prepared over the entire phase. In this manner, the components 
contained in the input signal to be analysed can be grasped properly on the bases 
by expanding the bases in temporal dimension, irrespective of the phase of 
components. Accordingly, there is an advantage that the property or feature of 
the temporally changing signal can be grasped correctly by considering the phase 
symmetry (temporal symmetry) in the bases, making the analysis of the signal 
more correct. 

[00022] Hence, in order to solve the above-mentioned problem, the present 
invention introduces phase symmetry (temporal symmetry) into the bases and 
considers the mixture of signals with time delay in accordance with Lewickis' 
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algorithm based on overcomplete representations, making it possible to separate 
the source signals from the mixture signals into which the source signals are 
mixed temporally and spatially, where the number of source signals is greater 
than or equal to the number of mixture signals. 

[00023] A principle of the invention will be described below. Firstly, the 
temporally and spatially mixed signals are formulated. It is supposed that the 
time series signal s(t) = {si(t), .., s n (t)} is generated from n signal sources. It is 
assumed that the time series signals Sj(t) (l<j<n) are independent of each other 
temporally and spatially and follow the same distribution. When these signals 
are mixed (convoluted) temporally and spatially, m signals x(t) = {xi(t), .., x m (t)} are 
obtained and expressed as follows. 

x iW = T SA..(T)5.(t~T) (formulas) 

j=l T--00 

Where l<i<m, and Aij represents the convolution filter. In Fig. 3, the convolution 
filters are represented in a network model. As shown in Fig. 3, for example, Xi(t) 
that is a component of x(t) is produced by applying the filters Aa, ..Aij, ..An to 
si(t), ..sj(t), ..Sn(t) that are components of the source signal s(t). 
[00024] In this invention, the temporally and spatially mixed signal 
represented in formula 8 is represented as a linear sum of bases such as the 
matrix model x = As represented in formula 1 in accordance with the Lewickis' 
learning algorithm. 

[00025] First of all, for the sake of simplicity, a case of m = 1 and n = 1 is 
considered. That is, the temporally and spatially mixed signal in which one 
source signal is convoluted as one mixture signal is considered. The subscripts of 
An, xi and si are omitted. From the formula 8, it follows that 
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x(t) = £ A(r)s(t-r) (formula 9) 
= Ait) * s{t) (formula 10) 
= ^At~ r)s(T) (formula 11) 

T=-co 

Herein, any consecutive (2q+l) points [x(-q), .., x(0), .., x(q)] among the infinite 
time series signal x(t) are formally represented in the matrix, using the formula 11, 
and given in the formula 12 and the formula 13 shown in Fig. 18. The matrix of 
the formula 13 indicates the matrix A in the formula 12. Also, it is supposed that 
the length of filter A(x) is 2q+l, and the value of A(x) is 0 in the range x<-q and x>q. 

That is, A(t) = [.., 0, 0, A(-q), .., A(0), A(q), 0, 0, ...] 
[00026] Herein, noting each column of the matrix A in the formula 13, the 
elements in each column are shifted by one point but composed of the same filter. 
Regarding each column as one basis function, each basis has phase symmetry 
(temporal symmetry). Therefore the temporally and spatially mixed signal x(t) is 
represented as a linear sum of bases, as shown in the model of Fig. 4. 
[00027] In this manner, considering the filter having the phase symmetry 
(temporal symmetry) as the basis, the signal mixed with time delay can be 
represented as the linear sum of bases. 

[00028] Herein, the matrix having the column in which filter Aij(x) = [Aij(-q), .., 
Aij(0), .., Aij(q)] is shifted in phase by one point as shown in the formula 13 is 
defined as a small matrix Ay in this invention. 

[00029] Though in the formula 12 and the formula 13, only one source signal 
s(t) is mixed with time delay, when n source signals {si, .., s n ) are mixed temporally 
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and spatially into m signal {xi, .., x m }, and formulated with the matrix, a linear 
model of the mixed signals is represented in the expression shown in Fig. 19. 
[00030] Herein, suppose that Xi(t) = [xi(-q), .., Xi(0), Xi(q)] (l<i<m) and Sj(t) = 
[sj(-2q), .., sj(O), .., sj(2q)] (l<j<n). {xi, .., x m } denotes m mixture signals and {si, .., 
s n } denotes n source signals. 

[00031] A combination of a plurality of matrixes and vectors contained in the 
formula 14 is integrated into one expression and formulated as an operation 
expression x = As. The matrix A in this operation expression is a basis matrix, 
and the small matrixes {Aij} composing the basis matrix A are basis small matrixes. 
The operation expression is applied to the Lewickis' learning algorithm based on 
overcomplete representations. Herein, the basis matrix A is represented as the 
expression shown in Fig. 20. 

And the mxn small matrixes (basis small matrixes) {Aij} composing the basis 
matrix A are represented as the expression shown in Fig.21. 
[00032] Since the temporally and spatially mixed signal Xi(t) as the input 
signal contains n filter components {Aa, Ai2, .., Ain) as shown in Fig. 3, it is 
expected that any of filter components {Aii, Ai2, Ain} is extracted for each of n 
basis small matrixes {Aij} (l<j<n) of the basis matrix A with the phase shifted by 
learning according to this invention. 

[00033] Herein, though the overcomplete basis matrix is usually learned in 
accordance with the formula 6, it is presumed that each column of mxn basis small 
matrixes {Ay} composing the basis matrix A is made up of one vector Arj(t) = 
[Aij(-q), .., Ay(0), .., Aij(q)], that is, each column of the basis small matrixes {A y } has 
phase symmetry. 
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[00034] If all the elements in the matrix are learned according to the invention, 
the shape of each column of the basis small matrixes {Aij} is different to collapse 
the phase symmetry (temporal symmetry) of bases. 

[00035] Thus, in this invention, the basis matrix A is learned according to the 
following procedure so that each column of the basis small matrixes {Aij} has the 
phase symmetry while the basis matrix A is corrected by learning, whereby the 
mixture matrix that is the optimal basis matrix A, and the source signals are 
estimated by this learning. 

(1) The temporally and spatially mixed signals Xi(t) are input, and the basis 
matrix A in which basis small matrixes{Aij} (l<i<m, l<j<n) having the phase 
symmetry of bases are gathered and Sj(t) are formulated as an operation 
expression as shown in the formula 14. 

Where Xi(t) = [xi(-q), .., Xi (0), .., Xi(q)] (l<i<m) and Sj(t) = [sj(-2q), .., Sj(0), .., Sj(2q)] 
(l<j<n). 

(2) The vector (basis) located in the (q+l)th column, namely, in the middle 
column, of the mxn basis small matrix {Aij} contained in the basis matrix A is 
corrected by learning. The basis after learning is supposed the (2q+l)-point 
vector [ A Aij(-q), .., A Aij(0), .., A Aij(q)]. 

More specifically, the basis matrix A is corrected for the vector (basis) in the 
middle column of the basis small matrixes {Aij} composing the basis matrix A in 
accordance with a learning rule AA in the formula 7. 

(3) The basis small matrixes are reconfigured on the bases after learning in the 
form of the formula 13. That is, the vector (basis) after learning in item (1) is 
shifted in phase successively to produce plural vectors (bases) having different 
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phase, and the new basis small matrixes { A Aij} which consist of the bases after 
learning and the bases having different phase produced after learning are formed. 

(4) The basis matrix A is reconfigured by arranging the basis small matrixes 
{ A Aij} in the form of the formula 14. The basis matrix A obtained thereby is 
regarded as the learned basis matrix. 

(5) The above (1) to (4) are repeated by changing the input temporally and 
spatially mixed signals, until the basis matrix A fully converges. And the 
converged value of the basis matrix A is calculated as the mixture matrix and the 
source signals are estimated. The estimated source signals *sj(t) (l<j<n) are the 
solution A s in the formula 2 when the basis matrix A has converged. 

[00036] As described above, with this invention, regarding the filter having 
phase symmetry as the basis, the mixed signal with time delay is represented as a 
linear sum of bases, and then the Lewickis' learning algorithm is applied. 
[00037] That is, the present invention provides a signal separation system for 
separating plural source signals before mixture from mixture signals having the 
source signals mixed temporally and spatially, comprising signal input means for 
inputting the mixture signals, signal storing means for storing the input mixture 
signals, and signal separation means for extracting the mixture signals stored in 
the signal storing means and separating the source signals from the extracted 
mixture signals, wherein the signal separation means transforms the extracted 
mixture signals into a linear sum of bases, and separates the source signals using 
a learning algorithm based on overcomplete representations. 
[00038] Also, this invention provides the blind signal separation system, 
wherein the signal separation means formulates the extracted mixture signals as 
an operation expression using a basis matrix composed of plural small matrixes 
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which consist of the bases with time symmetry around the time axis shifted, and 
calculates a mixture matrix to produce the mixture signals by mixing the source 
signals temporally and spatially to estimate the source signals to be separated, 
using the operation expression and the learning algorithm based on the 
overcomplete representations. 

[00039] Also, the invention provides the blind signal separation system, 
wherein on using the learning algorithm based on the overcomplete 
representations, the signal separation means calculates the mixture matrix and 
estimates the source signals to be separated by correcting a basis located in the 
middle column of the plural small matrixes in accordance with the learning 
algorithm based on the overcomplete representations. 

[00040] Also, the invention provides a signal separation method for separating 
plural source signals before mixture from mixture signals having the source 
signals mixed temporally and spatially, the method including a signal input step of 
inputting the mixture signals, a signal storing step of storing the input mixture 
signals, and a signal separation step of extracting the mixture signals stored at the 
signal storing step and separating the source signals from the extracted mixture 
signals, wherein the signal separation step includes transforming the extracted 
mixture signals into a linear sum of bases, and separating the source signals using 
a learning algorithm based on overcomplete representations. 
[00041] Also, the invention provides the blind signal separation method, 
wherein the signal separation step includes formulating the extracted mixture 
signals as an operation expression using a basis matrix composed of plural small 
matrixes which consist of the bases with time symmetry around the time axis 
shifted, and calculating a mixture matrix to produce the mixture signals by mixing 
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the source signals temporally and spatially and estimating the source signals to be 
separated, using the operation expression and the learning algorithm based on the 
overcomplete representations. 

[00042] Also, the invention provides the blind signal separation method, 
wherein on using the learning algorithm based on the overcomplete 
representations, the signal separation step includes calculating the mixture matrix 
and estimating the source signals to be separated by correcting a basis located in 
the middle column of the plural small matrixes in accordance with the learning 
algorithm based on the overcomplete representations. 

[00043] Also, the invention provides a blind signal separation program for 
enabling a computer to perform the blind signal separation method. 
[00044] Also, the invention provides a recording medium that records the blind 
signal separation program for enabling a computer to perform the blind signal 
separation method. 

[00045] With this invention, the source signals can be separated from the 
mixture signals into which the source signals are mixed temporally and spatially, 
where the number of source signals is greater than or equal to the number of 
mixture signals. 

[00046] The present invention realizes an object of separating source signals, 
the number of which is more than or equal to the number of mixture signals, from 
the mixture signals into which the source signals are mixed temporally and 
spatially in such a way as to formulate the mixture signals as an operation 
expression of a basis matrix and the source signals, namely, to transform the 
mixture signals into a linear sum of bases and apply a learning algorithm based on 
overcomplete representations. 
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[00047] Fig. 1 is a diagram showing the configuration of a system according to 
an embodiment of the invention. Reference numeral 1 denotes a blind signal 
separation system, 11 denotes a signal input portion for inputting the mixture 
signals into which the source signal are mixed temporally and spatially, 12 
denotes a signal storing portion for storing the input mixture signals, 13 denotes a 
signal separating portion for separating the source signals from the mixture 
signals by extracting the mixture signals from the signal storing portion 12, and 
calculating a mixture matrix based on the extracted mixture signals and 
estimating the source signals, and 14 denotes an output portion for outputting the 
calculated mixture matrix and the estimated source signals. 

[00048] The signal separation portion 13 comprises a formulation portion 131 
for formulating the mixture signals as an operation expression of a basis matrix 
and the source signals, a learning algorithm application portion 132 for applying a 
learning algorithm based on overcomplete representations as proposed by Lewicki 
et al., a mixture matrix calculating portion 133 for calculating the mixture matrix, 
and a source signal estimating portion 134 for estimating the source signals. 
[00049] That is, in the present invention, for example, the mixture signals are 
input into the signal input portion 11, and stored in the signal storing portion 12. 
And the formulation portion 131 within the signal separation portion 13 extracts 
the mixture signals stored in the signal storing portion 12, and formulates the 
extracted mixture signals as an operation expression of the basis matrix and the 
source signals. 

[00050] And the learning algorithm application portion 132 applies the 
learning algorithm based on overcomplete representations as proposed by Lewicki 
et al. The mixture matrix calculating portions 133 calculates the mixture matrix. 
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The source signal estimating portion 134 estimates the source signals. And the 
output portion 14 outputs the calculated mixture matrix and the estimated source 
signals. 

[00051] In a specific embodiment of the invention, three source signals are 
mixed in advance temporally and spatially to create two mixture signals, and the 
source signals are separated from the mixture signals, using the blind signal 
separation system 1 of the present invention. For example, three source signals 
are converted into mixture signals xi(t) and X2(t) by applying the filters An to A23, 
as shown in Fig. 5. 

[00052] In an embodiment of the invention, the basis small matrixes An to A m n 
in the formula 14 had all the dimension of 63x125 to make the experiment. That 
is, two 63-dimensional input mixture signals (xi(t),X2(t)) are prepared, so that 
three 125-dimensional output signals are obtained. Also, the standard deviation 
of the output signals by learning was normalized to 1, whereby a parameter 0 was 
set to V2. 

[00053] Three source signals in 10000 impulse signal train ([sj(l), 
Sj(10000)](l<j<3)) occurring at intervals according to a Poisson distribution with 
modulus 60 were prepared. The amplitude width was decided according to the 
uniform random number in a range [-1,1]. 

[00054] First 1000 points of each source signal are shown at the upper, middle 
and lower stages in Fig. 6. The longitudinal axis represents the amplitude width 
and the transverse axis represents the time. 

[00055] The mixture signals as input data were created by convoluting three 
source signals of Fig. 6 temporally and spatially in accordance with the formula 8. 
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The filters used for convolution are shown in Fig. 7. The longitudinal axis 
represents the amplitude width and the transverse axis represents the time. 
[00056] In Fig. 7, 200 denotes a filter An(t), 201 denotes a filter Ai 2 (t), 202 
denotes a filter Ai3(t), 203 denotes a filter A21U), 204 denotes a filter A22W, and 205 
denotes a filter A23(t). 

[00057] Also, first 1000 points of two mixture signals xi(t) and x 2 (t) obtained by 
convolution are shown in Fig. 8. The longitudinal axis represents the amplitude 
width and the transverse axis represents the time. xi(t) is shown at the upper 
stage and x 2 (t) is shown at the lower stage. 

[00058] Also, the initial value of basis is a vector of uniform random numbers of 
63 points generated in a range [-1,1], which is multiplied by a windowing function 
of Gauss distribution. This is intended to concentrate the power in the center, 
and extract the filter components as close to the center as possible. From 125 
basis functions obtained by shifting the phase every one point, 63x125 basis small 
matrixes {Aij} are formed. Thus, the initial value of the basis matrix A is obtained 
by preparing 2x3 kinds of the basis small matrixes and arranging them as in the 
formula 14. 

[00059] Fig. 9 is a flowchart showing a blind signal separation process 
according to an embodiment of the invention. First of all, the mixture signals into 
which the source signals are mixed temporally and spatially are input into the 
signal input portion 11 (step SI). For example, 63 consecutive points in the same 
time space are randomly extracted from two mixture signals xi(t) and X2(t) as 
shown in Fig. 8, so that the respective input data [xi(-31), .., xi(31)] and [x 2 (-31), .., 
X2(31)] are acquired, and input into the signal input portion 11 of the blind signal 
separation system 1. 
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[00060] Then, the formulation portion 131 formulates the input mixture signals 
(xi(t), X2(t)) as an operation expression using the basis matrix composed of plural 
small matrixes that are bases with time symmetry (step S2). For example, the 
initial value of the basis matrix A is obtained by forming 63x125 basis small 
matrixes {Aij} (l<i<2, l<j<3), and collecting 2x3 kinds of basis small matrixes, as 
described above, whereby the input mixture signals (xi(t), X2(t)) are formulated 
into the operation expression as shown in the formula 14. More specifically, they 
are formulated into the operation expression as shown in Fig. 10. 
[00061] Here, the basis small matrixes {Aij} are configured as shown in Fig. 11, 
for example. Also, the source signals Sj (l<j<3) in the operation expression as 
shown in Fig. 10 are configured as shown in Fig. 12, for example. 
[00062] Next, the learning algorithm is applied by the learning algorithm 
application portion 132 (step S3). For example, the learning algorithm 
application portion 132 applies the learning algorithm based on overcomplete 
representations as proposed by Lewicki et al. in the following way. 

(1) Using the operation expression as shown in Fig. 10, three output signals 
si(t), S2(t) and S3(t) are obtained by the linear programming method of the formula 
2. In an embodiment of the invention, a free software lp_slove is employed for 
computation in the linear programming. 

(2) The learning rule AA is computed from the output signals obtained 
according to the procedure (1), using the formula 6. 

(3) Using the formula 7, the bases are corrected. At this time, correction is 
made for the bases located in the middle column of the basis small matrixes {Aij} 
(l<i<2, l<j<3) composing the basis matrix A before correction, as previously 
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described. That is, correction is made for the bases located in the middle column 
of the basis small matrixes {Ay} as surrounded by the dotted line in Fig. 13. 

(4) Based on the bases after correction, the small matrixes are reconfigured in 
the form of the formula 13. That is, the new small matrixes { A Aij} are composed of 
the bases located in the middle column after correction and plural vectors with the 
bases successively shifted in phase. 

(5) The new small matrixes { A Aij} are arranged in the form of the formula 14 
and configured into the basis matrix as surrounded by the dotted line in Fig. 14. 
The basis matrix A obtained thereby is the learned basis matrix. 

[00063] Then, the learning algorithm application portion 132 determines 
whether or not the basis matrix A has fully converged (step S4). If the basis 
matrix A has not fully converged, the steps 1 to 3 are repeated by changing the 
input temporal and spatial mixture signals, until the basis matrix A fully 
converges. 

[00064] In an embodiment of the invention, for example, the steps 1 to 3 are 
repeated by 70000 times. The learning ratio r| in the formula 7 was set to 0.0005 
from the first time of input to 3000th time, 0.00025 from the 3001st to 20000th 
time, 0.00005 from the 20001st to 40000th time, and 0.00001 from the 40001st 
time and beyond. 

[00065] When the basis matrix A has fully converged, the mixture matrix 
calculating portion 133 calculates the converged value of the basis matrix A as the 
mixture matrix, and the source signal estimating portion 134 estimates the source 
signals (step S5). In an embodiment of the invention, the source signals are 
estimated as the solution of the formula 2 when the basis matrix A has become the 
converged value. As a result, three source signals (*sj(t) (l<j<3)) are estimated. 

-20- 



Finally, the output portion 14 outputs the calculated mixture matrix and the 
estimated source signals (step S6). Then, the procedure is ended. 
[00066] Fig. 15 shows the bases of the mixture matrix calculated in an 
embodiment of the present invention. In Fig. 15, 206 denotes the middle column 
of the basis small matrix An composing the calculated mixture matrix, 207 
denotes the middle column of the basis small matrix A12, 208 denotes the middle 
column of the basis small matrix A13, 209 denotes the middle column of the basis 
small matrix A21, 210 denotes the middle column of the basis small matrix A22, and 
211 denotes the middle column of the basis small matrix A23. The longitudinal 
axis represents the amplitude width and the transverse axis represents the time. 
They are estimated filters used when the source signals are convoluted. 
[00067] It is confirmed how correctly the source signals (*si(t),*S2(t),*S3(t)) are 
estimated from the mixture matrix composed of the basis small matrixes An to A23 
and the input mixture signals. 

[00068] Fig. 16 shows three source signals (estimated signals) estimated in an 
embodiment of the invention. The longitudinal axis represents the amplitude 
width and the transverse axis represents the time. When the signals at the upper, 
middle and lower stages in Fig. 16 are the estimated signal 1 (*si(t)), the estimated 
signal 2 (*S2(t)) and the estimated signal 3 (*S3(t)), the correlation coefficients 
between the source signals si(t), S2(t), S3(t) and the estimated source signals 
(estimated signals) in creating data are shown in Fig. 17. From Fig. 17, it can be 
found that the estimated signal 1, the estimated signal 2 and the estimated signal 
3 are highly correlated with the source signal S2(t), the source signal si(t) and the 
source signal S3(t), respectively. 
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[00069] Consequently, since the source signals were roughly estimated, it is 
confirmed that the blind signal separation system 1 of the invention actually 
enables the multi-channel blind signal separation. 

[00070] In an embodiment of the invention, the method for implementing the 
blind signal separation based on overcomplete representations has been proposed 
in which the temporal and spatial mixture signals are represented as a linear 
model by introducing the phase symmetry into the bases in the learning algorithm 
based on overcomplete representations. Also, by the computer experiments, more 
source signals than the number of mixture signals and the filters in the 
convolution could be estimated for the temporally and spatially mixed signals. 
[00071] Since proposed method presumes that there is no noise, it is required 
to add the noise term to the objective function in the formula 2, or take a measure 
of applying the algorithm of Olshausen et al. to minimize the square root error in 
practically dealing with the actual data. 

[00072] This invention is applicable to the uses in which the source signals, the 
number of which is more than or equal to the number of mixture signals, are 
separated from the mixture signals into which the source signals are mixed 
temporally and spatially, for example, the uses in which the source signals are 
separated from electromyogram signals or audio signals. 
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